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LONG-TERM  GOALS 

The  goal  of  our  work  is  to  develop  a  system  for  optical  communications  through  the  sea-air  interface 
and  to  provide  a  predictive  performance  model  for  the  system  for  different  environmental  conditions, 
source  power  levels,  receiver  range,  etc. 

OBJECTIVES 

The  scientific  objectives  of  the  project  involve  understanding  the  spectral  and  temporal  effects  of  water 
constituents  and  surface  characteristics  on  trans-interface  communication.  This  involves  the 
development  of  a  system  that  utilizes  several,  separated,  redundant,  sources  to  reduce  the  data  drop-out 
rate  due  to  wave  refraction.  Engineering  objectives  of  the  project  are  the  design  and  development  of  an 
intelligent  optical  receiver  that  can  identify  the  positions  of  redundant  sources  within  video  imagery 
and  control  a  micro  electro-mechanical  system  (MEMS)  device  to  direct  radiance  from  only  those 
sources  to  a  high-speed  sensor  for  demodulation.  This  spatial  filtering  should  reduce  interference  from 
surface  reflection  of  skylight. 

APPROACH 

Wave  refraction  of  light  rays  passing  through  the  sea-air  interface  causes  a  perturbation  in  the  path  of 
the  ray  relative  to  its  position  and  direction  for  a  flat  sea  surface.  While  gravity  waves  range  in  slopes 
up  to  about  15°  (e.g.  Lighthill  1980),  these  slopes  can  be  augmented  near  the  crests  by  parasitic 
capillary  waves  (Martin  2004).  According  to  the  Cox-Munk  (Cox  and  Munk  1954)  wave-slope 
relationship  to  wind  speed,  a  wind  of  lOm/s  results  in  a  mean  wave  slope  of  about  0.19.  Wu  (1990) 
suggests  a  more  rapid  increase  of  slope  with  wind  speed,  however.  Past  experience  in  trying  to  avoid 
sun  glint  when  performing  remote  sensing  operations  indicates  that  slopes  of  0.6  are  not  unusual  for 
individual  waves. 

For  capillary  waves,  the  disruption  in  directionality  of  an  optical  beam  or  ray  that  results  in 
information  dropouts  can  be  significant.  This  is  corroborated  by  an  experiment  made  in  Bayboro 
Harbor  (St.  Petersburg,  FL)  where  the  continuity  of  signal  from  a  single  LED  was  contrasted  against 
that  from  up  to  four  LEDs,  where  four  LEDs  were  separated  by  four  feet  at  the  comers  of  a  square 
frame  approximately  1  foot  below  the  interface.  Generating  capillary  waves  with  2  air  jets  provides 
enough  capillary  wave  activity  to  provide  significant  signal  drop-outs.  Redundant,  spatially  separated 
sources  are  used  with  digital  signal  processing  (DSP)  of  video  imagery  to  identify  the  positions  of 
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redundant  sources  within  the  imagery  (areas  of  interest,  AOIs).  Radiance  from  these  dispersed  sources 
it  then  collected  onto  a  common  intensified  receiver  to  reduce  interference  from  surface  reflection  of 
skylight  while  increasing  the  signal  for  high-rate  data  reception. 

BACKGROUND 

In  this  concept  for  optical  communication  through  the  air-sea  interface,  multiple  emitters  are  placed 
beneath  the  water’s  surface  and  imaged  from  above.  These  emitters  appear  as  small  points  of  light  in  a 
video  image  of  the  area.  In  a  project  funded  by  Blackbird  Technologies,  we  have  demonstrated  the 
ability  to  process  video  in  real-time  and  identify  our  emitters  within  the  video.  For  this  work,  we 
designed  optical  emitters  using  LEDs  that  transmit  data  using  on/off  keying.  Since  standard  video 
rates  are  used,  data  bandwidth  is  limited  to  5  baud  and  only  short  messages  containing  emitter  ID 
information  is  transmitted. 

Utilizing  digital  signal  processing  (DSP)  hardware,  we  developed  a  system  that  processes  NTSC  video, 
identifies  sources  within  the  video  matching  search  criteria,  decodes  data  transmitted  by  those  sources, 
and  outputs  the  input  video  signal  with  graphical  overlays  highlighting  the  detected  signals  and 
displaying  received  data.  Fig.  1  shows  the  prototype  electronics  (left)  and  a  screen  capture  from  a 
video  clip  (right).  The  screen  capture  shows  several  sources  of  light  imaged  through  a  narrow  band¬ 
pass  filter.  For  the  case  shown,  only  one  point  of  light  is  actually  a  device  (ID  399)  communicating 
data.  The  other  points  are  street  and  car  lamps  with  energy  in  the  pass  band  of  the  optical  filter. 
Without  our  DSP  approach,  these  other  points  would  not  only  decrease  S/N  but  could  be  identified  as 
spurious  targets.  We  have  demonstrated  the  ability  to  maintain  detection  of  signal  sources  under  a 
wide  range  of  environmental  conditions,  have  demonstrated  tracking  of  moving  signal  sources,  and 
proven  our  ability  to  distinguish  our  devices  from  sources  that  may  appear  similar  in  nature. 

This  extension  of  the  basic  system  involves  the  use  of  a  spatial  modulator  to  optically  direct  only  the 
portions  of  the  image  containing  signal  sources  to  high-speed  detectors  for  data  reception.  This 
concept  is  illustrated  below  in  Fig.  2.  Any  camera  outputting  NTSC  video  is  used  to  image  the  scene  of 
interest.  The  video  signal  is  processed  by  the  DSP  electronics  to  determine  the  coordinates  of  signal 
sources.  These  coordinates  are  used  to  control  an  active  modulator  to  direct  only  the  portions  of  the 
image  containing  signal  sources  to  a  high-speed  detector,  reducing  background  noise  and  raising  the 
signal-to-noise  ratio. 

MEMS  2-D  imaging  arrays  are  commercially  available  and  widely  used  in  TV  and  data  projection 
systems  [eg.  Texas  Instruments  Digital  Mirror  Device  (DMD)].  The  DMD  is  designed  to  tilt  each  of 
several  million  mirrors  into  1  of  2  positions  according  to  a  video  image  received  by  the  coupled 
electronics  module.  In  the  first  (default)  position  the  mirrors  are  turned  off  and  are  all  coplanar. 
Incoming  radiation  is  simply  reflected  at  the  same  angle  as  the  incoming  beam  (relative  to  the  surface 
normal). 


2 


Fig.  1:  Prototype  electronics  used  to  process  video  signals  to  locate  and  decode  optical  signals,  and  a 
representative  screen  capture  showing  the  detection  of  a  signal  source  placed  near  many  noise 

sources. 


Scene  (Filter)  Imaging 

Optics 


Fig.  2:  Actively-steered  high-speed,  high  sensitivity  detector.  2-D  Active  Array  (Eg.  TI DLP)  is  used 
to  actively  project  only  areas  of  the  scene  of  interest  onto  a  high  speed, 
highly-sensitive  PMT  detector. 


In  the  second  position,  the  mirrors  are  tilted  by  approximately  7  degrees.  Depending  on  the  orientation 
of  the  DMD,  the  reflected  beam  can  either  be  at  a  larger  or  smaller  angle  (by  14  degrees)  than  the  beam 
reflecting  off  the  mirrors  in  the  first  position.  By  turning  the  appropriate  mirrors  on  or  off,  light  from 
specific  portions  of  the  image  can  be  separated  from  other  regions.  It  is  proposed  that  such  a  2D  DMD 
array  be  utilized  within  a  custom  optical  system  to  allow  simultaneous  targeting  of  each  of  the 
individual  sources  that  comprise  the  underwater  transmitter.  Only  those  mirror  elements 
corresponding  to  the  locations  of  the  signals  will  be  energized  to  redirect  the  signals  to  the  high-speed 
detector.  As  wave  action  and  other  factors  interfere  with  the  reception  of  an  individual  source,  the 
corresponding  mirror  elements  will  be  turned  off  and  only  light  from  the  other  valid  sources  will 
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continue  to  the  high-speed  detector.  In  the  best  case,  all  individual  sources  will  be  present  and  their 
combined  energy  will  be  directed  to  the  detector  for  maximum  signal  to  noise.  Note  that  by  rejecting 
residual  background  image  noise  due  to  reflected  skylight  or  sun  glint,  the  gain  on  the  high-speed 
detector  can  be  increased. 

WORK  COMPLETED 

Scientific  A  single-wavelength  (525  nm),  4-source,  LED  array  was  constructed  and  deployed  in  both 
day  and  night  field  tests.  Analysis  of  the  data  acquired  during  these  deployments  indicates  that  the 
basic  hypothesis  is  correct  and  that  our  envisioned  approach  is  viable  (Carder  et  al.  2007).  Redundant 
sources  increased  transmission  efficiency  in  both  daytime  and  nocturnal  field  tests.  We  utilized  only 
narrow-band-pass  and  polarization  filters  on  the  camera.  Compared  to  losses  with  one  LED,  data  drop¬ 
outs  for  the  redundant  sources  were  reduced  by  factors  of  6-14  (see  Fig.  3). 


LED  Transmission  Efficiency  (Day) 


LED  Group 


Fig.  3.  Statistical  analysis  of  the  various  permutations  of  the  four,  redundant,  LED  sources.  Transmission 
efficiency  increased  to  95%  utilizing  all  four  sources  versus  30%  utilizing  a  single  LED. 


When  completed,  spatial  filtering  is  expected  to  increase  the  signal-to-noise,  and  thus  range  and/or  data 
rate,  by  a  factor  equivalent  to  the  ratio  of  the  total  image  area  divided  by  the  sum  of  the  LED- 
neighborhood  areas  of  the  images  (e.g.  >  20X). 

Engineering  Engineering  activities  have  focused  on  construction  and  testing  of  prototype  hardware 
to  demonstrate  proof-of-concept.  We  completed  construction  of  the  prototype  receiving  hardware  as 
shown  below  in  Fig.  4.  The  main  lens  views  the  target  and  produces  an  image  on  a  beam  splitter. 

One  path  from  this  beam  splitter  is  directed  towards  a  video  camera  for  image  display  and  processing. 
In  the  second  path,  an  enlarging  lens  relays  the  primary  image  onto  the  spatial  modulator  (DMD).  To 
compensate  for  the  different  sizes  of  the  camera  sensor  and  the  spatial  modulator,  the  relayed  image  is 
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magnified  by  a  factor  of  1 .6.  Light  reflecting  off  the  spatial  modulator  is  collected  and  focused  onto 
the  data  detector  for  subsequent  analysis. 


BEAM  SUTTB*  GAIA  Ufcl  bClOfl 


Figure  4:  A  photo  of  the  prototype  optical  receiving  hardware. 

Not  shown  in  the  figure  are  the  electronics  used  to  process  images  and  control  the  spatial  modulator 
(refer  to  Fig.  2).  In  our  concept,  optical  transmitters  will  appear  as  small  dots  in  the  images.  These 
transmitters  will  modulate  data  at  a  high  rate  (>  100  Khz)  within  a  much  slower  modulating  envelope 
(<  10  Hz)  that  can  be  detected  with  a  standard  video  camera.  Image  processing  will  detect  the  slow 
modulation  and  activate  only  those  elements  of  the  spatial  modulator  upon  which  the  modulating 
signal  is  incident.  The  active  elements  reflect  the  incident  light  to  the  data  (high-speed)  detector  for 
analysis.  The  goal  of  this  approach  is  to  increase  the  system’s  signal-to-noise  ratio  by  filtering  out 
ambient  scene  light  and  ultimately  detecting  only  that  light  received  from  the  close  neighborhood  of 
the  transmitted s). 

To  increase  gain,  we  replaced  the  photodiode  initially  chosen  for  the  data  detector  with  a 
photomultiplier  tube  (PMT).  These  detectors  offer  excellent  sensitivity  and  frequency  response  at  the 
cost  of  complexity. 

Spatial  Filter  This  experiment  demonstrates  the  effectiveness  of  using  a  Digital  Micro-Mirror  Device 
(DMD),  from  Texas  Instruments,  as  an  active,  digital,  spatial  filter,  allowing  optical  communication  in 
conditions  not  otherwise  possible. 

The  DMD  is  an  array  of  tiny  mirrors,  each  corresponding  to  one  pixel,  which  tilt  +-  12  degrees  along  a 
45  degree  axis  (lower-left  to  upper-right).  Using  an  optical  geometry  very  similar  to  that  used  in  video 
projectors,  we  were  able  to  use  the  DMD  to  reflect  light  from  specific  parts  of  the  image  onto  a  Photo- 
Multiplier  Tube  (PMT)  for  signal  detection,  or  reflect  light  away  from  the  PMT.  This  is  called  a 
“mask”  because  it  masks-off  most  of  the  image,  only  passing  the  parts  of  the  image  which  correspond 
to  the  LED  transmitting  the  signal.  The  light  that  is  reflected  away  from  the  PMT  is  called  the  “reject” 
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light.  This  light  can  be  viewed  with  a  camera,  and  we  call  that  the  “reject  image”.  By  observing  the 
reject  image,  we  can  verify  that  the  mask  corresponds  to  the  correct  part  of  the  image  because  the  hole 
in  the  mask  should  make  the  LED  invisible.  We  can  also  invert  the  mask  to  allow  our  camera  to  see 
the  light  that  should  be  reaching  the  PMT.  All  of  the  grayscale  images  below  were  captured  from  the 
reject  image  camera. 

TIOC  test  results  from  the  optics  lab  are  shown  in  Figure  5.  Equipment  was  in  a  light-tight  box. 
Fluorescent  room  lights  were  on.  Incandescent  lamps  were  off.  The  LED  emitted  850  nm,  high-power, 
light,  but  it  was  operated  using  a  large,  current-limiting  resistor  to  reduce  output  power.  No  optical 
band-pass  filter  was  used,  allowing  more  background  light  as  “noise”  in  the  image.  Imaging  was 
performed  with  a  3”,  400mm  spherical  mirror  directly  onto  the  DMD.  The  data  were  transmitted  from 
a  standard  computer,  RS-232,  serial  port  at  1 15200N81  (1 15,200  bits/second,  no  parity  bit,  8  data  bits, 
one  stop  bit)  through  an  IrDA  encoder  chip,  which  converts  each  bit  into  the  presence  (0)  or  absence 
(1)  of  luS  pulses.  After  reception  by  the  voltage-output  PMT,  the  signal  passed  through  a  single-pole 
100  kHz  band-pass  filter  with  a  gain  of  6.  After  this,  the  signal  was  passed  through  a  Schmitt  trigger 
and  a  555  chip  to  ensure  that  the  incoming  pulses  were  of  the  proper  width.  They  were  then  passed  into 
an  IrDA  decoder  chip,  which  output  RS-232  signals  to  the  serial  port  of  the  receiving  PC. 

The  lower  image  of  Figure  5  shows  the  received  signal  on  the  oscilloscope.  The  unfiltered  PMT  signal 
(pink)  is  off  the  screen  due  to  the  ambient  light  (noise).  Note  that  the  ambient  light  is  not  simply  a  DC 
offset,  or  else  it  would  be  filtered-out  by  the  100  kHz  high-pass  analog  filter  (yellow),  leaving  our  data 
signal.  The  oscilloscope  is  set  to  trigger  on  the  yellow  trace,  on  any  positive  pulse  shorter  than  3.8  uS 
and  as  tall  as  the  yellow  arrow  on  the  left.  One  may  observe  that  near  the  trigger  point  there  appears  to 
be  a  tendency  for  a  corresponding  square  wave  on  the  RS-232  output  (green  is  darker  on  the  upper  line 
and  darker  on  the  lower  line),  however  noise  is  causing  too  many  false  square  waves  to  differentiate 
the  data  signal  from  the  noise.  Despite  adjusting  the  oscilloscope  viewing  and  triggering  parameters, 
and  adjusting  the  PMT  gain,  no  usable  data  could  be  extracted  from  the  noise. 

In  the  two  images  in  Figure  5  the  image  is  focused  upon  the  DMD.  Without  using  the  DMD  as  a 
spatial  filter,  all  light  energy  in  this  image  reached  the  PMT.  Although  the  image  of  the  LED  appears 
obvious,  it  comprises  only  a  fraction  of  the  total  light  energy  in  the  image,  resulting  in  a  signal-to-noise 
ratio  much  less  than  1.  The  resulting  signal  is  indecipherable. 
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Figure  5.  Source  image  with  LED  “off”  (top  left)  and  “on”  (top  right);  oscilloscope  traces  for 

various  test  configurations  (bottom). 
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LED  “off’  and  masked  by  the  DMD 


LED  “on”  and  masked  by  the  DMD 
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LED  “off’  through  the  DMD  mask 

LED  “on”  through  the  DMD  mask 

Figure  6.  Mask  effects  on  the  images  rejected  by  the  DMD. 


In  the  upper  two  images  of  Figure  6,  we  can  see  the  “reject”  image  of  the  mask.  The  black  square  over 
the  LED  corresponds  to  the  DMD  mirrors  that  are  tilted  toward  the  PMT  and  away  from  the  camera. 
The  remaining  mirrors  are  tilted  toward  the  camera.  This  allows  us  to  view  the  image  on  the  actual 
DMD  and  verify  the  alignment  of  the  active  digital  spatial  filter  (mirrors  tilted  toward  the  PMT)  with 
the  signal  source  (LED).  The  lower  two  images  show  an  inverted  filter.  This  allows  us  to  view  the 
spatially  filtered  light  that  should  be  striking  the  PMT.  The  other  light  areas  in  the  image  (not  pointed- 
to  by  the  arrow)  are  noise,  possibly  due  to  glare  off  the  edges  of  the  mirrors  or  a  smudge  on  an  optic. 

The  effectiveness  of  this  technique  is  confirmed  in  the  oscilloscope  image  (Fig.  7)  showing  a  high 
signal-to-noise  ratio  (approximately  3: 1  in  the  raw  PMT  signal,  and  about  7:1  after  the  100kHz  high- 
pass  filter).  The  IrDA-encoded  pulses  are  easily  decoded  and  converted  back  into  RS-232  data,  as 
used  in  a  standard  computer  serial  port.  This  is  shown  in  the  green  trace.  The  character  received  here 
is  the  RS-232,  ASCII  encoded  letter  “f ’,  as  when  the  user  on  the  transmitting  computer  types  an  f  on 
the  keyboard  in  HyperTerminal.  In  this  encoding,  the  ASCII  code  for  “f  ’  is  01 1001 10,  and  RS-232 
specifies  a  start  bit  (0)  and  stop  bit  (1),  where  1  is  -7V  and  0  is  +7V.  IrDA  encoding  (yellow)  is 
similar,  but  it  uses  a  single  pulse  of  1.6uS  for  each  transmitted  0  and  the  absence  of  a  pulse  implies  a  1. 
There  is  a  slight  time  delay  in  the  decoding  from  IrDA  (yellow)  to  RS-232  (green). 


Figure  7:  Oscilloscope  showing  the  reception  of  the  letter  "f.  Pink  =  raw  PMT  signal.  Yellow  = 
output  of  100kHz  high-pass  filter  with  a  gain  of  6.  Green  =  RS-232  output. 


This  experiment  showed  the  active  digital  spatial  filter  to  be  very  effective  at  increasing  the  signal  to 
noise  ratio.  It  dropped  the  unfiltered  bias  in  the  image  due  to  background  light  (noise)  from  more  than 
5V  to  about  0.2  volts  after  spatial  filtration,  or  more  than  25X.  This  spatial  filtration  reduced 
background  light  enough  to  permit  the  system  to  transceive  data  nearly  flawlessly  at  the  high  data  rate 
of  1 12  kBaud.  Data  rates  were  mostly  limited  by  the  slew  rate  of  the  trans-impedance  amplifier  built 
into  the  PMT.  Additional  signal-to-noise  improvements  will  surely  be  seen  with  the  use  of  an  optical, 
narrow  band-pass  filter,  matched  to  the  wavelength  of  the  transmitting  LED.  For  use  outside,  both  the 
band-pass  filter  and  a  “full-power”  LED  would  be  used  to  obtain  more  range  under  daylight 
conditions. 

CONCLUSIONS 

Redundant  LED  sources  at  the  comers  of  a  1  -m  square  used  for  optical  communication  from  below- 
water  to  above  water  reduced  the  data  drop-out  rate  by  6-  to  14-fold  because  wave  facets  rarely 
directed  the  radiance  from  all  four  LEDs  away  from  the  receiver. 

Laboratory  tests  of  elements  of  optical  communication  in  a  light-rich  environment  (e.g.  reflected 
skylight)  were  also  performed.  Because  narrow-band-pass  (NBP)  optical  filters  are  well-understood  in 
terms  of  suppressing  out-of-band  background  light,  NBPs  were  not  used  in  the  laboratory.  Instead, 
tests  were  of  a  spatial-filter  to  restrict  the  field  of  view  of  a  high-gain  receiver  to  the  neighborhood 
about  the  light  source.  This  approach  reduced  the  background  light  25-fold,  enabling  the  PMT  to  be 
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operated  at  a  much  higher  gain.  The  resulting  signal-to-noise  ratio  was  about  3:1.  After  passing  the 
spatially  filtered  PMT  signal  through  a  high-pass  electronic  filter,  the  signal-to-noise  ratio  was 
increased  to  7: 1 .  This  was  sufficient  to  allow  optical  communication  at  1 12  kBaud.  In  transitioning 
outside  during  daylight,  the  LED  emitter  will  be  returned  to  full  power,  and  a  narrow  band-pass  filter 
will  be  added  to  contend  with  the  brighter  environment. 

IMPACT/APPLICATIONS 

There  are  many  scenarios  where  communication  between  a  submerged  asset  and  an  airborne  asset 
would  be  very  desirable.  However,  communication  through  the  air/sea  interface  is,  at  best, 
problematic,  and  any  high-bandwidth  communication  strategy  remains  elusive.  Tests  of  our  basic 
approach  (redundant,  separated  sources  with  a  spatial  filter  of  the  image  field  of  view)  have  shown  that 
our  hypotheses  were  sound.  Data  dropouts  were  reduced  by  factors  from  6X  to  14X,  and  background 
light  rejection  was  about  25X.  This  provides  enough  signal  to  noise  to  permit  optical  data  transmission 
rates  up  to  112  kBaud.  Additional,  brighter  LED  sources  will  further  reduce  data  drop-outs. 

RELATED  PROJECTS 

•  USF  College  of  Marine  Science  Center  for  Underwater  Observability  and  Optical 
Communication  -  Utilization  of  the  ROSEBUD  ROV  and  the  TOPO-13  air  platform 

•  Eric  Kaltenbacher,  SRI  International  -  Optical  tagging  and  3D  underwater  imaging. 
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